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(54) Image and data display method and apparatus 



(57) An image and data display method includes the 
steps of: determining a display condition (18) having dis- 
play and non-display of relative data on the basis of a 
previously determined user identifier and utilization con- 
dition (17); displaying a picture and a character string 
indicative of contents of a least one relative data deter- 
mined to be able to be displayed in the previous step on 
the basis of the display condition together with the image 
during display of a specific scene; displaying, when at 



least one of pictures and character strings is selected, 
relative data corresponding to the selected picture and 
character string on the basis of the display condition; 
and displaying, when a command for displaying a list is 
inputted, a list of picture and character strings displayed 
until now and, when at least one in the list is selected, 
displaying relative data corresponding to the selected 
picture and character string on the basis of the display 
condition. 
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Description 

The present invention relates to a method and an 
apparatus for displaying an image and data obtained 
from broadcasting or the like and more particularly to an 
image and data display method and apparatus for dis- 
playing data classified or individualized for each user in 
synchronism with an image. 

Further, the present invention relates to a method 
of providing information (hereinafter referred to as con- 
tents) on a network together with information of goods 
such as advertisement or the like and more particularly 
to a method of providing the information of goods clas- 
sified or individualized for each user in relation to the 
contents. 

As a service of broadcasting data such as charac- 
ters and pictures besides images, a character broad- 
casting using terrestrials waves and a data delivery 
service using a satellite or a CATV are started now. In 
future, when broadcasting would become digital, a large 
number of data can be transmitted at the same time as 
transmission of images by the data compression tech- 
nique and the data multiplexing technique and accord- 
ingly it is considered that these data broadcasting serv- 
ices arc increased extensively. 

The displaying method in the conventional data 
broadcasting is now described in order. 

First, in the character broadcasting, data are multi- 
plexed in an unoccupied area of surface waves to be 
transmitted and electric waves received by a receiving 
terminal are divided into images and data. Since the da- 
ta is not related to the images, the data is separated 
from a display screen of the images and is displayed. 
For example, as shown in Fig. 8, a display screen is di- 
vided into two portions so that an image is displayed in 
one of the portions and data is displayed in the other. It 
is general that images and data are displayed in a dis- 
play screen alternately by a remote control operation. 
There is also a display method that when reception of 
the data is completed while an image is viewed, the 
completion is displayed on the image as a telop. Since 
a temporary memory area is also provided in the termi- 
nal, there is a way to use that the received data is stored 
temporarily to be viewed later. 

In any event, since the data is independent from the 
image, the data can be viewed at any time regardless 
of the image from the time of completing the reception 
of the data. 

Further, in the service named the interactive televi- 
sion, data related to an image is transmitted together 
with the image. The data is multiplexed in the unoccu- 
pied area of the terrestrials waves to be transmitted in 
the same manner as the character broadcasting. When 
data related to a scene at that time arrives at the termi- 
nal, a specific icon is displayed at a comer of the image 
to notify a viewer of the arrival. When the viewer desig- 
nates display of the data, the display screen is divided 
into two portions as shown in Fig. 8 and the image and 



the data are displayed at the same time. There is also 
a method that an image and data are displayed on one 
display screen alternately. 

On the other hand, recently, a television (named an 

5 Internet television) capable of viewing a home page (ex- 
plained below) on the Internet is being born into the 
world. The display method thereof is substantially iden- 
tical with the character broadcasting and the home page 
is displayed without quite synchronizing with an image. 

10 Generally, the display method thereof involves the 
method of displaying a television image and the home 
page on two display screens at the same time as shown 
in Fig. 8 and the method of displaying the television im- 
age and the home page on one display screen alternate- 

75 |y. Further, in recent television programs, an address of 
a home page is often informed to the viewers, while it is 
necessary that the user inputs the address in order to 
view the home page by means of the Internet television. 
The prior art pertinent to the interactive television 

20 and the Internet television is described in "The Age of 
Digital Televisions Has Come !", Nikkei Trendy, '95, 
Oct., pp. 54-73 issued by Nikkei Home Co. in detail. 

Next, there is the World Wide Web (WWW) on the 
Internet of providing contents on a network. In the 

2S w\N\N t contents provided therein are named the home 
page. The home page is generally described by the 
script language named the HTML (Hyper Text Markup 
Language). Further, in the HTML, the relation among the 
home pages are defined as the hyper link. The user des- 

30 ignates a WWW server and a home page by a unified 
format named the URL (Uniform Resource Locator). 
When the WWW server receives a request from the us- 
er, the WWW server transmits a proper home page to 
the user. 

35 in a general method of providing information of 
goods in the WWW, advertisement (information of 
goods) relative to the home page is displayed in the 
same home page. Since the information of goods rela- 
tive to the home page required by the user is provided, 

-to power of appealing to the user so that the user buys 
goods is increased. Generally, the home page and the 
advertisement often correspond to each other in ad- 
vance. 

An information service considered by the present 
invention is below described by using a definite exam- 
ple. 

Recently, with improvement of the performance of 
hardware and the diffusion of network, the interactive 
information service can be realized easily. The interac- 
so tive information service mentioned here is the informa- 
tion service having the bidirectional characteristics that 
not only an information provider sends information to the. 
user one-sidedly but also the user accesses to the. in- 
formation provider positively. For* example, the conven- 
es tional television broadcasting and the radio broadcast- 
ing are one-sided media from a television program pro- 
vider to viewers and listeners, while the facsimile and 
telephone which are bidirectional media are used to re- 
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alize the interactive information service such as a viewer 
participating program. Further, an experimental televi- 
sion program employing the Internet and the communi- 
cation using other communication methods instead of 
the facsimile and telephone is also started. Henceforth, 5 
it is considered that the information service using the 
television broadcasting and the Internet combined with 
each other is extensively increased conjointly with the 
boom of the Internet 

In such a bidirectional information service, an effec- 
tive and efficient inquiry from the information provider to 
the users is important. Unless an inquiry is first made by 
the information provider, interaction is not also started. 
Considering the merit of the information provider, it is 
desirable that large effects are attained by the inquiry 
with reduced labor. 

Considering how to provide information effectively, 
information (which is sometimes named as data since 
the information is contrasted with images) is sent togeth- 
er with images and accordingly it is desirable that both 
of the information and the images have any relation. 
There are considered utilization methods in which, for 
example, in a television program a questionnaire on the 
program is sent simultaneously or more detailed infor- 
mation of goods is sent in a television commercial. 
When a television terminal capable of accessing to the 
Internet or a CD-ROM is supposed, an automatic ac- 
cess program to an address of a home page or data in 
the CD-ROM may be merely sent. In this manner, when 
the access method from the image to the information is 
defined, there is a merit that the viewers are attracted 
by the image and move to the bidirectional media im- 
mediately. At the same time, the viewers can know de- 
tailed information pertinent to the image easily. 

As a method of providing information effectively, 
there is considered that information suitable for each in- 
dividual user is sent directly. For example, in a current 
television commercial and television shopping, the 
same image information is provided to all of the users 
uniformly. However, when it is realized that information 
of goods having the appealing power in fact can be pro- 
vided to each individual user together with images on 
the basis of customer information (character and taste 
of users), the efficiency of marketing is improved re- 
markably. The customer information includes charac- 
ters and taste of the users. At the same time, since in- 
formation suitable to the user's Interest is sent to the 
user, the efficiency of reference to the information is also 
increased. Such individualization technique is named 
mass customization or one-to-one marketing. Further, 
when different information is sent to each user, the in- 
formation provider's load is also increased and a load 
of the network is also heavy. Accordingly, the users arc 
classified by common characteristics so that information 
is provided to each classified user. The concept "classi- 
fication" is also contained in the mass customization. 

Problems in case where the above-mentioned in- 
teractive information service is applied to the conven- 
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tional image and data display method are now de- 
scribed. 

First, in the character broadcasting, relation is not 
specifically defined between images and data. Similarly, 
it is not premised that relation exists between television 
images and home pages on Internet television. Accord- 
ingly, there was no problem even if the images and data 
were displayed in different display screens independ- 
ently. Further, in the character broadcasting, even if the 
method of displaying data on the image was performed, 
it was sufficient only by notifying that preparation of dis- 
play was completed after reception of data had been fin- 
ished. However, in the information service pertinent to 
the present invention, since data relative to the images 
is treated, it is necessary to notify the users which scene 
of the images the data corresponds to. 

On the other hand, data relative to images is dis- 
played together with the images in the current interac- 
tive television. Accordingly, the display method in which 
a scene of the images to which the relative data corre- 
sponds is notified to the user is used. However, since it 
is not a display method in which the mass customization 
is premised and the same data is delivered to the users, 
data to be viewed in synchronism with the images is all 
the same for the users. 

It is an object of the present invention to solve the 
above problems in order to realize the above information 
service by providing an image and data display method 
in which the users are first notified that data classified 
or individualized for each user exists in synchronism 
with images and the data is displayed in synchronism 
with the images. 

Further, in order to cause the users to access to rel- 
ative data, existence of the relative data must be under- 
stood more easily. Since it is considered that a plurality 
of data are also sometimes related to a specific image 
scene, measures for identifying differences among the 
plurality of data easily are required. Accordingly, it is an- 
other object of the present invention to provide an image 
and data display method of notifying to users which data 
is related to the image plainly. 

In addition, since the images are media continuing 
in terms of time, the users have a desire to view the im- 
age continuously without disturbing the image by any 
obstacle. Accordingly, it is another object of the present 
invention to provide an image and data display method 
which ensures that relative data is provided to users 
without losing the continuity of images. 

Furthermore, since the interactive information serv- 
ice described above aims at common families, it is nec- 
essary that the users' terminals can be operated simply 
and used easily. Accordingly, it is another object of the 
present invention to provide an image and data display 
method which is simple to operate and easy to use. 

Further, in order to spread the interactive informa- 
tion service into common families widely, it is indispen- 
sable that the users' terminals are inexpensive. Accord- 
ingly, it is another object of the present invention to pro- 
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vide an image and data display method which can real- 
ize inexpensive terminals. 

Further, in the interactive information service, it is 
important that an information provider's intention is sat- 
isfied sufficiently so that information is given to the user 
of the target exactly, for example. Accordingly, it is an- 
other object of the present invention to provide an image 
and data display method which can reflect the informa- 
tion provider's intention. 

Next, in the service of providing information of 
goods in the WWW, generally, the home page and the 
advertisement are often provided to the user in one-to- 
one correspondence manner. Accordingly, the service 
of providing an classified or individualized advertise- 
ment to each user cannot be realized as it is. 

It is another object of the present invention to pro- 
vide and display information of goods classified or indi- 
vidualized for each individual user in relation to contents 
while the user refers to the contents. Particularly, since 
there is a case where information of a plurality of goods 
is selected and provided to a target user at the same 
time, the present invention relates to a method of solving 
the confliction state thereof. 

According to the present invention, a method of dis- 
playing an image and data relative to a specific scene 
of the image, comprises (a) the step of selecting, when 
one of a plurality of images is selected, at least one data 
relative to the image simultaneously, (b) the step of de- 
termining a display condition including display and non- 
display of the relative data on the basis of a previously 
determined user identifier and utilization condition, (c) 
the step of determining, when a plurality of relative data 
are determined to be able to be displayed in the step 
(b), display priorities among the relative data, (d) the 
step of displaying a picture and character string indica- 
tive of contents of each relative data on the basis of the 
display priorities determined in the step (c) together with 
the image during display of the specific scene, (e) the 
step of displaying, when a picture and character string 
is selected, relative data corresponding to the selected 
picture and character string on the basis of the display 
condition, and (f) the step of displaying, when a com- 
mand for displaying a list is inputted, a list of pictures 
and character strings displayed until now and, when at 
least one in the list is selected, displaying relative data 
corresponding to the selected picture and character 
string on the basis of the display condition. 

In the steps (a) and (b), the data relative lo the im- 
age being displayed and customized for the user is se- 
lected. Further, in the step (d), the user can be notified 
by the picture and the character string indicative of con- 
tents of the relative data that the relative data exists in 
the image scene. In addition, in the step (e), the relative 
data can be displayed in synchronism with the image in 
response to the user's request. 

Moreover, in the step (b), the display condition of 
the picture and the character string is determined on the 
basis of the utilization condition such as a size of the 



display terminal of the user. Similarly, in the step (e), 
when a plurality of relative data exist in the same image 
scene, the display priority such as the order of displaying 
the relative data is determined. In step (d). since the pic- 
s ture and the character string are displayed on the basis 
of the display priority and the display condition, the ex- 
istence of the relative data can be notified to the user 
plainly. 

Further, in the step (f), when the user inputs a com- 
io mand of displaying a list, a list of pictures and character 
strings displayed until now is displayed. The user can 
select any one in the list and refer to the relative data. 
That is, the user can first view only the images continu- 
ously and refer to the relative data slowly later. Since 
is there is provided the mode that the picture and the char- 
acter string cannot be displayed, the mode can cope 
with the need that the user wants to view only the picture 
on the whole image. 

Furthermore, in the present invention, in the step 
(e), the whole display screen for displaying the relative 
data is divided into at least one image display area and 
at least one data display area. There are provided a 
mode for simultaneously displaying the image and the 
data relative to the image and a mode for displaying at 
least one relative data on the whole display screen. 
When a command for changing over the display is in- 
putted, the display mode is changed among the image 
display mode, the image and data simultaneous display 
mode and the data display mode. The user can select 
the mode in accordance with the user's purpose. In ad- 
dition, since the operation of selecting the mode can be 
made by the button of the remote controller, there can 
be provide the function that the operation is simple and 
easy. 

Further, in the present invention, in step (f), the 
number of relative data displayed in the list is limited and 
selection of relative data is made in accordance with the 
display priority when the limitation is exceeded. 

Accordingly, the memory area can be maintained to 
be small and consequently a cost of the terminal can be 
reduced. 

Moreover, in the present invention, there is provided 
the mode that the information provider can forcedly dis- 
play the picture and the character string indicative of 
contents of the relative data. Even if the user sets the 
picture and the character string of the relative data not 
to be displayed, the picture and the character string are 
necessarily displayed for the relative data in the forced 
display mode and accordingly the intention of the infor- 
mation provider can be satisfied sufficiently such that 
information can be transmitted to the user of the target 
exactly. 

In the information -of -goods providing method ac- 
cording to the present invention, when the user selects 
contents, information of goods relative to the contents 
is also selected at the same time and whether the infor- 
mation of goods is displayed or not is determined for 
each user on the basis of the display condition of the 
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information of goods. When a plurality of information 
items of goods can be displayed by the above process- 
ing, the display priorities are determined among the in- 
formation items of goods. Further, during display of the 
contents, data indicative of the contents of the informa- 
tion of goods is displayed together with the contents on 
the basis of the display priorities determined by the 
above processing. In addition, there is provided a mem- 
ory area for temporarily storing all information of goods 
selected for each user and including the information of 
goods which cannot be displayed during reference of 
the contents by the user since the display priority thereof 
is low and the temporarily stored information of goods 
can be displayed in accordance with the user's request. 
In the drawings 

Fig. 1 is a schematic diagram illustrating a function- 
al block of an image and data display method and 
apparatus according to an embodiment of the 
present invention; 

Fig. 2 is a schematic diagram illustrating a system 
configuration of an interactive information service 
according to an embodiment of the present inven- 
tion: 

Fig. 3 is a schematic diagram illustrating a hardware 
configuration of an image and data display method 
and apparatus according to an embodiment of the 
present invention; 

Fig. 4 is an explanatory diagram showing display 
pictures of an image and data display method and 
apparatus according to an embodiment of the 
present invention; 

Fig. 5 is an explanatory diagram showing different 
display pictures of an image and data display meth- 
od and apparatus according to an embodiment of 
the present invention; 

Fig. 6 is an explanatory diagram showing different 
display pictures of an image and data display meth- 
od and apparatus according to an embodiment of 
the present invention; 

Figs. 7A, 7B and 7C are explanatory diagrams 
showing different display pictures of an image and 
data display method and apparatus according to an 
embodiment of the present invention; 
Fig. 8 is an explanatory diagram showing a display 
picture in a prior art image and data display method 
and apparatus; 

Fig. 9 is a schematic diagram illustrating an external 
appearance of a remote controller of an image and 
data display method and apparatus according to an 
embodiment of the present invention; 
Fig. 10 is an explanatory diagram showing concep- 
tually a construction of transmitting images and da- 
ta simultaneously; 

Fig. 11 is an explanatory diagram showing concep- 
tually another construction of transmitting images 
and data simultaneously; 

Fig. 12 is an explanatory diagram showing a data 



model in an image and data display method and ap- 
paratus according to an embodiment of the present 
invention; 

Fig. 13 is an explanatory diagram showing a data 
5 structure of an image and data display method and 
apparatus according to an embodiment of the 
present invention; 

Fig. 1 4 is an explanatory diagram showing an over- 
all processing of an image and data display method 
10 and apparatus according to an embodiment of the 
present invention; 

Fig. 1 5 is an explanatory diagram showing a flow of 

processing of a display change-over unit; 

Fig. 1 6 is an explanatory diagram showing a flow of 

75 processing of an image display unit; 

Fig. 1 7 is an explanatory diagram showing a flow of 
competition solving processing; 
Fig. 1 8 is an explanatory diagram showing a flow of 
overlay processing; 

20 Fig. 19 is a schematic diagram illustrating an overall 
system in case where the present invention is ap- 
plied to a method of providing advertisement in 
WWW on the Internet; 

Fig. 20 illustrates a display picture of a WWW 
2S browser in a user terminal; 

Fig. 21 is a flow chart showing processing in a con- 
tents server; 

Fig. 22 is a flow chart showing processing in a user 
terminal; 

30 Fig 23 is an explanatory diagram showing a data 
structure of user individual information; 
Fig. 24 is an explanatory diagram showing a data 
structure of an advertisement list; and 
Fig. 25 is a diagram showing a conceptual structure 

35 of a home page. 

An embodiment of the present invention is now de- 
scribed with reference to the accompanying drawings. 
Fig. 2 illustrates a system configuration of an inter- 
ne active information service according to the present in- 
vention. The system includes a subsystem 102 on the 
side of the information provider, an information trans- 
mission media 101 and a user terminal 1 . As information 
transmission media 101, broadcasting media such as 
terrestrials waves, satellite broadcasting 103 and CATV, 
network media such internet 104 and package media 
such as CD-ROMs 105 and DVD (Digital Video Disk) 
are considered. In the subsystem 102 on the side of the 
information provider, a module 109 processes various 
50 contents 110 such as images and data into structure 
suitable for the transmission media and defines links 
among the media. The links among the media means, 
the reference relation among information sent to the us- 
ers by means of the transmission media, "For example, 
55 access can be changed from an image produced by 
means of broadcasting media to data on a home page 
on the Internet or a CD-ROM related to the image. In- 
formation is sent from a broadcasting station 106 to the 
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satellite broadcasting 103, from an Internet server 107 
to the Internet 104, and from a package preparing sub- - 
system 108 to the CD-ROM 105. The user terminal 1 
receives the information to refer thereto. 

Fig. 1 is a functional block diagram schematically 5 
illustrating of the user terminal 1 . The user terminal 1 
includes a body apparatus 2, a display unit 3 and user 
input means 4. The body apparatus 2 receives at least 
one image source 5 and at least one data 6. A transmit- 
ting method of image and data and a structure of data 
are described later. 

The body apparatus 2 can be divided broadly into 
a functional block 10 for selectively receiving an image 
and data relative to the image, a functional block 11 for 
displaying the image and the relative data in synchro- 
nism with each other, and a functional block 12 for ac- 
cessing to an external resource such as the internet and 
the CD-ROM. First, the functional block 10 selects at 
least one image in an image selection unit 13 on the 
basis of a command from the user input means 4. The 
selection is the same as usual selection of a channel in 
a television, while the present invention is characterized 
in that data relative to the image is selected at the same 
time as the selection of the image. For example, when 
it is assumed that the user selects an image A of the 
channel number 1 , data a and b relative to the image A 
are selected. Then, the image is sent through an image 
input unit 14 to an image and data combination unit 22 
of the functional block 11. 

On the other hand, the relative data is sent through 
a data input unit 15 to a data selection unit 16. The data 
selection unit 16 of the present invention is character- 
ized to further select relative data on the basis of previ- 
ously determined utilization condition 17. For example, 
when it is assumed that the user is a man of 30 years 
old and the utilization condition 17 is registered to that 
effect, the relative data sent to the data selection unit 16 
is compared with the utilization condition, so that the da- 
ta a is selected. The condition of a plurality of kinds such 
as address, hobby, character and the like besides age 
and sex can be registered as the utilization condition. At 
the same time since an identifier of each individual user 
is also registered, the identifier of the user is set in the 
utilization condition on the side of the relative data as 
information to be sent toonly a particular user. With such 
structure, the information provider side can narrow the 
users down to provide information effectively and at the 
same time since the user side can refer to only data rel- 
ative to the user himself truly, the efficiency of reference 
to information is increased. 

At the same time, when the condition concerning 
utilization environment is registered, the display condi- 
tion 18 of relative data is prescribed on the basis of the 
condition. For example, processing that a display area 
of the relative data is conformed to a size of the display 
screen is considered. Further, actually, the method that 
the display condition is prescribed for each utilization 
environment condition and the user terminal side se- 
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lects the prescribed condition is supposed such that the 
relative data conforms to a display condition a under a 
condition A. 

It is necessary that the functional block 11 displays 
an image and data so that the user can understand the 
relation of the image and the data. Accordingly, the func- 
tional block 11 provides display means of three kinds 
including an image display unit 26, a combination dis- 
play unit 27 and a data display unit 28. Examples of pic- 
tures by the display means are described in detail with 
reference to Figs. 4 to 7. In brief, the image display unit 
26 displays an image on the whole display screen main- 
ly. The combination display unit 27 displays an image 
and data in combination. The data display unit displays 
only data on the whole display screen. Further, display 
screens are changed over by means of a display 
change-over unit 29 in response to a command from the 
user input means 4. 

As far as the image display unit 26 outputs only an 
image to the display unit 3, the relation of the image and 
data cannot be transmitted to the user. Accordingly, the 
present invention is characterized by the provision of a 
synchronization control unit 21 and the image and data 
combination unit 22 in order to display data relative to a 
specific scene of an image by an icon or telop indicative 
of contents thereof on the image. Particularly, the image 
and data combination unit 22 includes data overlay 
processing 24 for combining to superpose an icon or te- 
lop on an image. The icon is used to show a kind of the 
relative data. For example, the icon represents that the 
relative data is present information, limited information 
for members or the like. Further, the telop represents a 
headline of the relative data. It is desirable that contents 
of the telop can be understood at a glance like a big 
headline of a newspaper. 

At the same time, a data editing unit 20 is provided 
in order to edit the icon and the telop in advance on the 
basis of the display condition 18 when the display con- 
dition 18 is prescribed in the relative data. 

For example, when a size of a display screen is 
small, a size of the icon and the telop is also make small 
similarly and a length of the telop is made short. As de- 
scribed above, the icon and the telop can be used to 
notify the existence of the data relative to the image to 
the users plainly and effectively. 

Through processing 25 in the image and data com- 
bination unit 22 displays an image as it is without super- 
posing the icon and telop on the image. The through 
processing 25 is used when there is no data relative to 
the image or when the user turns off display of the icon 
and telop. 

The combination display unit 27 displays the image 
and the relative data in combination. The combination 
display method includes a method of superposing the 
image on the relative data by image overlay processing 
23, a method of superposing data on the image by the 
data overlay processing 24, a method of dividing a dis- 
play screen of the display unit 3 into an image display 
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area and a data display area and the like. Examples of 
pictures are described with reference to Figs. 4 to 7. The 
present invention is characterized in that when relative 
data is present in a specific scene of the displayed im- 
age, the icon and telop are displayed to be superposed 
on the image in the same manner as the image display 
unit 26 and when the user inputs a command for data 
display, relative data is displayed in the data display ar- 
ea. There is a merit that the user can view images of 
continuous media continuously without disturbing the 
images by displaying data and can refer to relative data 
on the same display screen simultaneously. 

The data display unit 28 displays only relative data. 
In this case, when the display condition 1 8 is prescribed, 
the data editing unit 20 processes and edits data so that 
the data conforms to the condition. The functional block 
1 1 includes a data temporary memory area 1 9, in which 
relative data selected by the data selection unit 16 is 
stored temporarily. The relative data is sent to each 
processing unit in response to a request from each dis- 
play unit. 

Relative data is described briefly. In the present in- 
vention, as the relative data, characters and images 
having a so-called link structure are supposed like a 
home page on the Internet. Operation on a data display 
picture corresponds to operation of following links. Ac- 
cordingly, the functional block 12 includes a data oper- 
ation unit 32 for designating a link destination and a data 
search unit 33 for searching for the link destination. Fur- 
ther, the present invention is characterized in that rela- 
tive data is not only sent but also taken out from trans- 
mission media such as the Internet 104 and the CD- 
ROM 105. Accordingly, the data search unit requests to 
search not only the data temporary memory area 19 but 
also an Internet access unit 34 and a CD-ROM access 
unit 35 for data in accordance with information relative 
to an access method of relative data. The Internet ac- 
cess unit 34 receives an URL (Uniform Resource Loca- 
tor) of a home page on the internet 1 04 and obtains the 
home page corresponding to the URL to store it in the 
data temporary memory area 19. The CD-ROM access 
unit 35 has substantial identical processing, while since 
there is a possibility that an access method is different 
for each CD-ROM title, it is necessary to previously de- 
termine an access protocol to the CD-ROM used in the 
information service. In the embodiment, a home page is 
described as a representative of contents of data rela- 
tive to images hereinafter. 

Fig. 3 schematically illustrates a hardware of the us- 
er terminal 1 . The user terminal 1 includes the body ap- 
paratus 2, the display unit 3 and the user input means 
4. Further, the body apparatus 2 includes a central con- 
troller 1 1 1 , a memory unit 1 1 3, a display control unit 11 4, 
an input control unit 115, a broadcasting receiving unit 
1 1 6, a disk control unit 1 1 7 and a communication control 
unit 118 connected to one another through a bus 112. 
The memory unit 113 includes the data temporary mem- 
ory area shown in Fig. 1 and stores a processing pro- 



12 

gram and data therein. The display control unit 114 con- 
trols output to the display unit 3. The input control unit 
115 receives the command inputted from the user 
through the user input means 4 and sends the command 
s to the central controller. The central controller interprets 
contents of the command and executes an instruction 
for realizing the command. The broadcasting receiving 
unit 116 receives images and data sent from a broad- 
casting station through an antenna 1 1 9. The disk control 
unit 1 1 7 accesses to data in package media such as the 
CD-ROM 15 or the like. Similarly, the communication 
control unit 118 accesses to various information sources 
such as home pages on the internet 104 and the like. 
Figs. 4 to 7 show the transition of display pictures 
1$ of an embodiment according to the present invention. 

The transition of pictures is described with refer- 
ence to Fig. 4 as an example. First, only an image is 
displayed in a display picture 200. When a time stamp 
is started during display of the image, the picture chang- 
es to a display picture 201 or 203. Incidentally, the time 
stamp is a structure for relating a specific image scene 
to data relative to the scene and represents a time from 
a beginning of the related image scene to the termina- 
tion thereof. When any command is not inputted by the 
user during the time stamp, the display picture 201 or 
203 is returned to the display picture 200. 

An icon 202 representative of a kind of related data 
is displayed in the display picture 201 and a telop 204 
representative of contents of related data is displayed 
together with the icon. The icon 202 and the telop 204 
are included in the related data. The present invention 
is characterized in that the icon and the telop notify that 
there is data related to the image scene. Further, in the 
present invention, since data classified or individualized 
for each user is displayed, it is characterized that the 
icon or the telop is used to notify a kind or contents of 
data to the user plainly. In the embodiment, whether the 
telop is displayed or not can be set by the information 
provider. In addition, there is a mode in which the user 
can turn off display of the icon and the telop so that the 
user enjoys the image without disturbing the image by 
the icon or the telop. However, there is related data 
which is desired to be necessarily notified to the user 
depending on contents of the related data and accord- 
ingly there is provided a forced display mode of the icon 
and the telop. 

Next, in the state of the display picture 200, when 
the user instructs to change over the display, the display 
is moved to a display picture 205, so that a home page 
viewed lastly or a default home page is displayed. On 
the other hand, when the user instructs to change over 
the display in the state of the display picture 201 or 203, . 
the display is moved to a display picture 21 3 in which a 
home page 214 relative to the image is displayed. As 
described above, the embodiment Is characterized in 
that when the change-over of the djsplay is instructed 
during display of the icon or telop, the home page rela- 
tive to the image scene is displayed. 
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Further, in the display pictures 205 and 213, a re- 
duced image 207 of a currently selected image is dis- 
played to be superposed on the home page. As de- 
scribed above, the present invention is characterized in 
that the user can view the image continuously without $ 
disturbing the image by display of data and at the same 
time the user can refer to the relative data. 

In addition, the reduced image 207 usually has the 
same function as the display picture 200 of the image. 
Accordingly, in the display picture 205, when a time to 
stamp j£ started, the display picture is moved to a dis- 
play picture 209 in which a telop 211 and an icon 212 
are displayed on a reduced image 210. In this state, 
when the user instructs to change over the display, the 
display picture is moved to the display picture 213 in is 
which a home page synchronized with the image is dis- 
played. In this manner, since the user can be notified 
that there is new data relative to the image during simul- 
taneous display of the image and the relative data, the 
user can refer to the relative data immediately 20 

Next, when the display is instructed to be changed 
over again in the state of the display picture 205, the 
display picture is moved to a display picture 21 5 in which 
the reduced image 207 disappears from the picture and 
only the currently displayed home page is displayed. 25 
Similarly, when the display is instructed to be changed 
over in the state of the display picture 209, the display 
picture is moved to a display picture 218 in which the 
reduced image 210 disappears and at the same time a 
home page synchronised with the image is displayed. 30 
This display mode is suitable for the case where the user 
wants to view the home page without disturbing it by the 
image on the contrary to the combined display mode of 
the image and the home page. Incidentally, operation of 
the home page is made by a menu 208. Further, when 3$ 
the display is instructed to be changed over again, the 
display picture is returned to the display picture 200 in 
which only the image is displayed. 

The menu 208 has an item of "re-display of auto- 
matically preserved icon and telop". That is, icons and 40 
telops viewed by the user until now are automatically 
preserved as a list and when there is an indication from 
the user, the list is converted into a home page format 
(that is, tormat of HTML (Hyper Text Markup Language) 
to provide it to the user. For example, when the indica- •*£ 
tion to that effect is inputted in the display picture 215, 
the display picture is moved to a display picture 217. A 
list of icons and lelops automatically preserved until now 
is displayed in the display picture 217. In the present 
invention, it is supposed that there is a need that the so 
user overlooks relative data while the user is full of en- 
thusiasm about the image and the user wants to view 
the relative data again later, and the above function of 
displaying the list of icons and telops is provided to cope 
with such need. Further, when all of icons and telops are 55 
preserved, the capacity of the memory unit 113 be- 
comes insufficient at any memory capacity. Accordingly, 
the priority is given to the relative data so that unneces- 



sary data can be eliminated from the data having the 
lower priority. For example, when it is assumed that the 
priority is determined in accordance with the cost re- 
quired for provision of information, the data is eliminated 
in order of a lower cost. 

As described above, the transition of pictures has 
been described with reference to Fig. 4. There are three 
kinds of display pictures including the display mode of 
only the image, the combined display mode of the image 
and the home page and the display mode of only the 
home page. In Fig. 4, the modes are changed in order 
of the image display mode, the combined display mode 
and the home page display mode and return to the im- 
age display mode. On the other hand, Fig. 5 shows a 
different example of display pictures of the present in- 
vention, in which the modes are changed in order of the 
image display mode, the home page display mode and 
the combined display mode. The transition of pictures 
of Fig. 5 is suitable for utilization that the home page is 
viewed mainly. Similarly, Fig. 6 also shows a different 
example of display pictures of the present invention, 
while the picture example does not contain the com- 
bined display mode. In order to display the combined 
image and home page, it takes cost correspondingly. 
However, the picture example of Fig. 6 can suppress the 
cost low. 

Further, as the combined display mode of the image 
and the data, there are some other picture examples as 
shown in Figs. 7A, 7B and 7C. For example, in Fig. 7A, 
a display screen 230 is divided into two areas. One of 
the areas is defined as an image display area 231 and 
the other is defined as a home page display area 232. 
In the image display area 231 , existence of a relative 
home page is notified by an icon 234 and a telop 233. 
At this time, when the user instructs to change over the 
image, a home page 235 synchronized with the image 
is displayed in the home page display area 232. 

In Fig. 7B, a display screen 240 is divided into one 
home page display area 241 and three image display 
areas 242, 243 and 244. The user selects an image from 
the image display areas and when data relative to the 
image exists, the existence is notified by an icon 245 
and a telop 246. A home page synchronized with an im- 
age B can be displayed in response to an input com- 
mand for changing over the display. 

Fig. 7C shows a display example in which it is as- 
sumed that there are a plurality of data relative to a spe- 
cific image scene. An image 251 is displayed now and 
a list 252 of relative data such as titles of relative data 
is displayed. When the user selects relative data from 
the list, a pertinent home page 253 is displayed. As an- 
other display method in the case where there are a plu- 
rality of data relative to a specific image scene, there is 
a display method in which in the 'display picture 201 or 
203 of Fig. 4 the display time of the icon and telop is 
time-divided among the relative data or all of icons and 
telops of the relative data are arranged on the picture. 

Further, in Figs. 4 to 7A, 7B and 7C, it is premised 
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that the existence of relative data is notified to the user 
and the user selects data to be displayed. When the im- 
age and the relative data can be viewed on the same 
picture like the display picture 209 of Fig. 4 and the dis- 
play pictures of Figs. 7 A, 7Band 7C t there is a utilization 
form that display of the relative data is also changed au- 
tomatically when a time stamp of the image is changed. 

Fig. 9 shows a remote controller 260 of an example 
ot the user input means in the present invention. The 
remote controller 260 includes a power button 261, a 
channel change-over button 262, a volume change but- 
ton 264, a channel direct selection button group 263 and 
the like in the same manner as a usual television remote 
controller. The present invention is characterized in that 
the remote controller 260 includes a display change- 
over button 265 and a home page display button 266. 
The display change-over burton controls the transition 
of states by the change-over of display in the display 
pictures of Fig. 4. That is, when the user pushes the dis- 
play change-over button, the display picture moves 
among the display pictures ot Fig. 4. Further, in Fig. 4, 
the transition from the display picture 209 to the display 
picture 213 and the transition from the display picture 
201 or 203 to the display picture 218 are performed by 
the homo page display button 266. 

Further, in the embodiment, since processing for 
designating a link destination in the home page is re- 
quired, cross keys 257, 268, 269 and 270 for moving a 
cursor up and down and right and left and a decision 
button 271 are provided. In the embodiment, the cross 
keys are used not only to designate a link destination in 
the home page but also to select relative data desired 
to be viewed by the user among a plurality of data rela- 
tive to a specific image scene, for example, when the 
relative data are displayed as a list as shown in Fig. 7C. 
The cursor may be moved by different means other than 
the cross keys. Further, selection may be made without 
using the cursor. 

Fig. 10 shows arrangement of simultaneous trans- 
mission of image and data in terrestrials wave. Images 
are usually sent for each image frame 280 in surface 
wave, while an area 281 named a VBI (Vertical Blanking 
Interval) for taking synchronization in the vertical direc- 
tion is provided between frames. Recently, it is being 
permitted that empty portion of this area can be used for 
the data broadcasting. Data 282 are embedded in the 
area 281 in the broadcasting station and the data are 
taken out in the user terminal. 

Fig. 11 shows arrangement of data broadcasting in 
the digital satellite broadcasting. 

In the common analog satellite broadcasting, one 
transponder of a satellite is used to transmit one chan- 
nel. However, in the digital satellite broadcasting, the im- 
age compression technique and the data multiplexing 
technique can be used to transmit images for four pro- 
grams per transponder. At the same time, since some 
empty areas can be produced, this areas can be used 
to transmit data to the user terminal. At present, this ar- 



eas are used to transmit a program guide or the like. 

Fig. 12 illustrates a data model of relative data in 
the present invention. Data 291 holds a utilization con- 
dition 291 which prescribes users who can refer to the 
5 data. At the same time, the data has a time stamp 293 
to be able to specify a related image scene 292. Further, 
the data holds an icon 294 and a telop 295 displayed to 
be superposed on an image in order to notify the exist- 
ence of relative data to the user. At the same time, the 
data holds data contents 296 or an access method to 
data entity. 

The above data structure is materialized as shown 
in Fig. 13. That is, data is managed by an identifier (id) 
number and includes an image identifier (id), a utilization 
condition, a starting time, an end time and a duration of 
a time stamp, an image expressive of an icon, a char- 
acter string displayed as a telop, and data contents. The 
image identifier (id) represents an identifier of a program 
image and uses a G code, for example. Further, an im- 
age scene related to data can be specified by the image 
identifier (id) and the starting time and the end time of 
the time stamp. As the utilization condition, an attribute 
value is set for each of previously determined attributes. 

For example, the priority for selecting one of a plu- 
rality of relative data corresponding to one imago is also 
included as one of the attributes. Alternatively, when the 
data is individualized, an identifier of the user is stored 
in this area. With regard to data contents, there is a pos- 
sibility that the data is present in a home page on the 
Internet or a CD-ROM. In this case, an access method 
to the data is stored in the area. 

A flow of processing for implementing the image 
and display method of the present invention is now de- 
scribed with reference to a PAD diagrams shown in Figs. 
14 to 18. Programs corresponding to processing shown 
in Figs. 14 to 18 can be stored in a memory medium 
such as a floppy disk and be read in a memory upon 
start of the programs to be executed. The memory me- 
dium may store the programs and is not limited to a flop- 
py disk. 

Fig. 1 4 shows a processing flow of a main program 
300 in the image and data display method. First, in step 
301 , it is detected that the user pushes the power button 
261 of the remote controller 260 to turn on a switch. In 
step 302, a channel used when the switch was turned 
oft last time is displayed. This processing in step 302 
corresponds to processing in the image display unit and 
is described later. In step 303, processing in steps 304 
and 305 is repeated until the switch is turned off. In step 
304, a button of the remote controller 260 operated by 
the user is detected. In step 305, processing corre- 
sponding to each button is performed. When the display 
change-over button 265 is selected, processing 306Jn 
the display change-over unit is performed* 

Next, when the home page display button 266 is se- 
lected, home page display processing 307 is performed. 
In the home page display processing, the mode is 
moved to the data display mode when the existence of 
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relative data in the image display mode is notified by an 
icon and a telop in Fig. 4 and data synchronized with the 
image is displayed. Similarly, in the combined display 
mode, when a telop and an icon are displayed in the 
reduced image picture and the existence of relative data $ 
is notified to the user, the display is changed over to a 
home page synchronized with the image in the com- 
bined display mode. Next, when the channel button 262 
or the channel direct selection button group 263 is se- 
lected, processing 308 in the image selection unit 13 is io 
performed. The present invention is characterized in 
that data relative to the image is also changed over dif- 
ferently from change-over of the channel of the usual 
television. 

A data operation button means the cross key having >5 
the buttons 267 to 271 . When the data operation button 
is selected, processing 309 in the data operation unit is 
performed. For "example, the processing includes 
processing that a link destination on the home page is 
designated. Next, when the power button 261 is select- 20 
ed, the program 300 detects switching off and escapes 
from a loop 303. In step 311, a channel upon switching 
off is recorded so as to be able to display a television 
program immediately upon switching on next time. 

In step 312, the program is terminated. Operation 25 
such as, for example, adjustment of volume is consid- 
ered else, while since such operation departs from the 
gist of the present invention, it is omitted. 

Fig. 15 shows a processing flow 306 in the display 
changeover unit 29 First, in step 321, a display mode 30 
at the present time is detected. In the embodiment, as 
shown in Fig. 4. there are three kinds of display modes 
including the image display mode, the combined display 
mode of image and data and the data display mode. In 
step 322, when the current display mode is the data dis- 35 
play mode, processing 302 in the image display unit 26 
is performed and the mode is moved to the image dis- 
play mode. When the current mode is the image display 
mode, processing 323 in the combined display unit 27 
is performed and the mode is moved to the combined -to 
display mode. When the current mode is the combined 
display mode, processing 324 in the data display unit 
28 is performed and the mode is moved to the data dis- 
play mode. Finally, in step 325, the processing 306 is 
terminated. is 

The processing 324 of the data display unit in the 
embodiment is processing for displaying a home page. 
In the processing 323 of the combined display unit 27, 
display of a home page relative to an image scene is 
prepared on a rear side of the image upon change-over so 
of the picture. In this case, when there is no relative data, 
display of a finally viewed home page or default home 
page is prepared. After the completion of preparation, 
the image display area is reduced successively and at 
the same time the data display area is viewed to the user ss 
successively. The reason why animation or the like is 
used such that the image display area is reduced suc- 
cessively is that it is prevented that the user feels that it 



is difficult to understood since display of image and data 
is divided in a moment. 

Fig. 1 6 shows a processing flow 302 in the image 
display unit 26. The method of Fig. 10 is considered as 
the arrangement of the simultaneous transmission of 
image and data. Since data is stored for each frame of 
the image, it is necessary to take out the data for the 
frame unit. Steps 331 to 344 are repeated every prede- 
termined period (in the embodiment, every one frame 
unit) in step 330. 

First, in step 331 , one frame image is inputted in the 
image input unit. In next step 332, data embedded be- 
tween frames is extracted. Since it is considered that 
data is not embedded, whether data is present or not is 
judged in step 333 and when data is present, steps 334 
to 338 are performed. Processing in step 334 corre- 
sponds to processing of the data input unit 15 and in 
step 334 data is extracted to be decoded. Processing in 
step 335 corresponds to processing in the data selection 
unit 16 and in step 335 the registered utilization condi- 
tion is collated with the utilization condition in the data. 
In next step 336, when both the utilization conditions are 
coincident with each other, the data is stored in the tem- 
porary memory area 19 in step 337. When both the uti- 
lization conditions are not coincident, the data is not 
stored in step 338. In step 335, the display condition of 
data such as a size of the icon and a font size of the 
character string is prescribed on the basis of the condi- 
tion of utilization environment such as a size of picture 
screen. Since an amount of data received in one frame 
unit is not so much, operation that data is divided to be 
sent is considered. In this case, it is necessary to com- 
bine data divided for each segment to unify the data after 
the data preserving processing 337. 

Processing in next step 339 corresponds to 
processing in the synchronization control unit 21 and in 
step 339, data corresponding to the current time zone 
in the currently displayed television program is re- 
trieved. When the data structure of Fig. 1 3 is examined, 
the identifier (id) of the image and the time stamp are 
also previously prescribed and accordingly the identifier 
of the image and the time are compared with the iden- 
tifier of the currently displayed image and the current 
time. In step 340, when coincident data are present, 
steps 341 to 342 are performed and when coincident 
data are not present, step 343 is performed. In step 341 , 
since there is a possibility that a plurality of data are se- 
lected at Ihe same lime, processing for solving such 
confliction is required. After solving the confliction, in 
step 342 the icon and the telop are overlay-displayed 
on the image. Each processing is described in detail lat- 
er. 

Thereafter, the displayed icon and telop or the icon 
and telop which have low dispiay'priority and cannot be 
displayed are recorded. These icons and telops are con- 
verted into a home page in response to a request from 
the user and are prepared to be always accessed. In 
step 343, since there is not pertinent data, the image is 



10 

BNSDOCtD: <EP 0851681A1 I > 



19 



EP 0 851 681 A1 



20 



outputted to the display unit. In step 344. the combined 
image of icon and telop or the image is displayed for one 
frame unit. Processing in steps 339 to 343 may be re- 
peated every predetermined period. The processing in 
steps 339 to 343 is not necessarily required to be per- 
formed for each frame, while in the embodiment the 
processing is embedded in the loop 330 of one frame 
unit. When the mode is changed to the data display 
mode or when the switch is turned off, the processing is 
terminated in step 345. 

Fig. 17 shows a processing flow of confliction solv- 
ing processing 341 . In step 350, whether any other data 
relative to the same time zone is present or not is ex- 
amined. In the embodiment, the case where a plurality 
of data are assigned in the same time zone is named 
the competition. As shown in Fig. 7C, when a list of rel- 
ative data can be displayed in the form of list, the list of 
relative data may be displayed as it is, while when an 
icon and a telop indicative of contents of each individual 
relalive data are displayed, it is necessary to decide an 
order of display, a display time and the like. Thus, wheth- 
er there is a confliction or not is detected in step 351 and 
when there is no competition, the processing is termi- 
nated as it is (step 352). When there is a confliction, the 
display priority of data is determined and scheduling is 
prepared again so that the icon and the telop are dis- 
played in order of the priority. Finally, the processing 341 
is terminated in step 354. 

Fig. 18 shows a processing flow of overlay display 
processing 342. In step 360, a display mode of icon and 
telop by the user and a forced display mode of icon and 
telop by the information provider are detected. When the 
forced display mode is on even if the display mode of 
icon and telop is off, data and a telop are forcedly dis- 
played on the image to be overlapped on each other. 
Accordingly, in order not to perform the overlay process- 
ing, it is necessary to turn off the display mode of icon 
and telop and also turn off the forced display mode by 
the information provider. In step 362, the icon and the 
telop are individualized for each user on the basis of the 
previously determined display condition. In next steps 
363 and 364, when the telop is present, processing 364 
for superposing the telop on the image is performed. 
Similarly, in step 365, when the icon is present, process- 
ing 366 tor superposing the icon on the image is per- 
formed. The processing method of superposing a char- 
acter string and a picture on the image conforms to the 
processing for superposing data on usual image to be 
displayed. 

The embodiment of the present invention has been 
described laying stress on the broadcasting media, 
while the present invention can be used as the image 
and data display method using package media and 
communication media. 

Next, a second embodiment in which the present 
invention is implemented in the utilization form in which 
contents on a network are provided together with infor- 
mation of goods such as advertisement or the like is de- 



scribed. More particularly, the present invention is ap- 
plied to a method of providing information of goods in 
the WWW on the internet. 

Fig. 1 9 is a schematic diagram illustrating an overall 

5 system of the embodiment. The system includes a con- 
tents server 400 and a user terminal 410 both of which 
are connected to the internet 415. 

The contents server 400 includes a WWW server 
401 and an advertisement management program 403, 

io which are interlocked with each other by means of a CGI 
(Common Gateway Interface) conforming to the stand- 
ard of WWW. 

The WWW server 401 provides a home page 402 
in response to a request from the user terminal 410. As 

is described above, in the embodiment, the home page 
constitutes contents. 

The advertisement management program 403 
manages advertisement and provides information 404 
of goods suitable for the user. More particularly, an area 

20 (hereinafter referred to as an advertisement area) in 
which the advertisement can be displayed is previously 
ensured in the home page 402 and suitable advertise- 
ment is embedded in the area. Further, as described lat- 
er, since user's individual information is sent from the 

2S user terminal 41 0, suitable advertisement is selected on 
the basis of the user's individual information. At this 
time, when a plurality of information items of goods are 
selected, the display priority of each advertisement is 
prescribed and the advertisement is displayed in ac- 

30 cordance with the priority. 

The user terminal 410 includes a WWW browser 
41 1 , a timer management program 41 2, and an individ- 
ual information management program 413. The timer 
management program 412 and the information man- 

35 agement program 41 3 prescribe inter-program interfac- 
es with respect to the WWW browser 41 1 and are inter- 
locked with the browser. Alternatively, it is considered 
that the WWW browser 411 includes the timer manage- 
ment program 412 and the information management 

40 program 413, while the embodiment deals with the 
above structure. 

The WWW browser 411 requires the home page 
from the WWW server 401 and receives the home page 
sent from the WWW server to be displayed. 

45 The timer management program 41 2 manages the 
duration of display of each home page and sends a dis- 
play request command of next information (in the em- 
bodiment, particularly, information of goods) to the 
WWW browser 411 every predetermined time. When 

50 the WWW browser 41 1 can cope with the request by the 
fact that information pertinent to the request is stored in 
a cache or a buffer, the WWW browser 411 copes with, 
to that extent and when it is necessary to make inquire 
to the server side, the WWW browser sends a command 

55 to the server. " ■ . 

The individual information management program 
411 manages the user's individual information 413 and 
sends the individual information to the WWW server 401 
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through the WWW browser 411 if necessary. Since the 
WWW server 401 selects information of goods on the 
basis of the individual information, the individualized or 
classified information of goods is sent to the user. 

The individual information is information concerning s 
characters such as age, sex, address and the like and 
taste such as hobby, liking and the like of the user. There 
is considered implementation including, for example, (1) 
when the WWW browser makes inquiry to the WWW 
server, a network address of the user terminal is in- to 
formed to the WWW server and accordingly this address 
is used as the individual information, (2) a file or program 
for taking custody of and managing the individual infor- 
mation is prepared separately, and (3) the individual in- 
formation is managed by a third party agency collective- *s 
ly and the WWW server makes inquiry to the agency on 
the basis of the user's identifier. Particularly, in the 
above items (2) and (3), in many cases, the user previ- 
ously decides the individual information which has no 
problem if it is opened to the public in view of protection 20 
of privacy. In the embodiment, the implementation (2) is 
supposed to make description. 

The utilization image or outline of the embodiment 
is now described. 

Fig. 20 shows a picture configuration of the WWW 25 
browser 411 in the user terminal 410. The WWW brows- 
er 411 includes a window 420 having a menu 421. The 
window includes an advertisement area 422 for display- 
ing an advertisement (information of goods) 423 and a 
home page display area for displaying contents required 30 
by the user Definitely, there is considered implementa- 
tion such as (1) the window is divided into an area in 
which a home page by one HTML file is displayed and 
an area in which an advertisement is displayed, and (2) 
the advertisement 423 is also set as one home page 35 
and a frame tag of the HTML is used to display a plurality 
of home pages in one window. In the embodiment, the 
implementation (2) is supposed make description. Fur- 
ther, in the case of (2), a parent home page for deciding 
an arrangement of the home page in the window is re- 10 
quired, while it is assumed that the home page including 
the contents has the function of the parent home page 
and the home page for the advertisement is included in 
the home page for contents. 

Generally, the advertisement 423 is often named a -*$ 
burner advertisement, while the present invention is 
characterized in that a plurality of advertisements can 
be displayed in respeel to one home page and even if 
the home pages are identical, the advertisement is indi- 
vidualized or classified for each user. In other words, in so 
Fig. 20 even if contents displayed in the home page dis- 
play area 424 are identical, there is a possibility that con- 
tents displayed in the advertisement area 422 are dif- 
ferent for each user. Further, a plurality of advertise- 
ments are assigned to the home page 424 and the ad- 55 
vertisement 423 in the advertisement area 422 is 
changed to another advertisement every predetermined 
time. 



As methods of displaying a plurality of advertise- 
ments, there is considered implementation such as (1) 
the duration for displaying one advertisement is provid- 
ed as described above and advertisement is changed 
every duration, and (2) when the advertisement area 
422 is sufficient, the advertisements are displayed as a 
list. In the embodiment, the implementation (1) is sup- 
posed to make description. 

A flow (Fig. 21 ) of processing in the contents server 
is now described. 

First, the WWW server 401 receives a content re- 
quest command from the WWW browser 411 (step 430). 
Then, the command is analyzed to retrieve pertinent 
contents (step 430). As described above, in the embod- 
iment, the home page constitutes the contents. Further, 
since the advertisement area is previously provided in 
the home page and advertisement is embedded in the 
advertisement area by later processing, the retrieved 
contents are preserved temporarily. 

Then, in step 432, the WWW server 401 obtains the 
individual information from the WWW browser 411. It is 
considered that the individual information is sent from 
the WWW browser together with the content request 
command, while generally the WWW server receives a 
content request command and request the individual in- 
formation to the WWW browser. 

Fig. 23 shows a data structure of the individual in- 
formation. The individual information includes attribute 
values 462 for each user defined for previously pre- 
scribed attributes 460. The individual information is 
managed as arrangement including a combination of at- 
tributes 460 and attribute values. The attributes 460 in- 
clude characters such as age, sex and address and 
taste such as hobby, favorite music genre, favorite mov- 
ie genre and kinds of favorite books of the user. The at- 
tributes are not required to be unified for each service 
and can be prescribed on the side of the user or the serv- 
ice provider newly. Further, since conventional or fixed 
information may be selected as the characters in input- 
ting of the attribute values, a structure such as a pull- 
down menu which is easy to select the attribute values 
is adopted. In addition, a simple structure is adopted so 
that the taste can be inputted by giving questionnaires 
to the user and responding to the questionnaires. 

Next, the information-of-goods management pro- 
gram 403 retrieves information of goods coincident to 
the user's individual information and preserves the co- 
incident information of goods temporarily (steps 433 to 
436). 

A list of advertisements as shown in Fig. 24 is used 
to retrieve information of goods. In the list of advertise- 
ment, the user's attribute of a target is prescribed for. 
each advertisement (470 to 477 of Fig. 24). items (480 
to 486 of Fig. 24) of the attributes are the* same as the 
individual information of Fig. 23. Further, since the at- 
tribute values increase a width of the target user, a range 
or a plurality of attribute values are often inputted. For 
example, in the advertisement 470, the attribute of age 
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480 is prescribed to a range from 20 to 35 years old. In 
Fig. 24, "Don't Care" means that the attribute value may 
be any value. That is, as the retrieval condition, refer- 
ence is not made to the item of "Don't Care". Further, 
although not shown in Fig. 24, logical symbols such as 5 
"NOT", 'AND" and "OR" may be used to prescribe com- 
plicated conditions for each item. 

In step 433, an advertisement corresponding to the 
contents is first retrieved from the list of advertisements. 
As shown in Fig. 24, since the list of advertisements in- 
clude an item 487 in which URLs of home pages can be 
recorded, a pertinent advertisement is retrieved on the 
basis of the URLs. Since an advertisement relative to 
contents can be displayed by causing contents to cor- 
respond to advertisements, there is a merit that the ap- 
pealing power of goods to the user can be increased. It 
is a matter of course that it is considered that specific 
contents do not correspond to advertisement. 

Next, a vector matching to the received individual 
information of the user is taken for each advertisement. 
In the vector matching, each advertisement and individ- 
ual information are both regarded as sets (vectors) of 
attribute values and the degree of matching is calculated 
between the two vectors. As the calculation method, 
there are methods as follows: (1) whether the attribute 
value of the individual information is coincident with the 
condition of the advertisement list or not is calculated 
for each attribute item and the calculated results are AN- 
Ded, and (2) to what extent the attribute value of the 
individual information is coincident with the condition of 
the advertisement list is calculated for each attribute 
item and the calculated results are combined. In the em- 
bodiment, the method (1 ) is supposed to make descrip- 
tion. 

In step 434, when the advertisement is coincident 
with the individual information, the advertisement is tem- 
porarily preserved (step 435) and when the advertise- 
ment is not coincident with the individual information, 
operation is moved to step 436. The display priority of 
the advertisement preserved temporarily in step 435 is 
determined later. In step 436, the processing is repeated 
for a next advertisement. 

There are many cases where there are a plurality 
of advertisements coincident with the user's individual 
information. For example, in the case of Figs. 23 and 
24, the advertisements 470 to 477 are coincident with 
all of the user's individual information. As described 
above, since the display area of advertisement has a 
limitation, all of advertisements cannot be displayed at 
a time. Accordingly, in the present invention, the display 
priority is prescribed and advertisements are changed 
successively in accordance with the priority. 

In step 437, the display priorities among the tempo- 
rarily preserved information of goods are decided. As 
definite methods, the following methods are considered: 
(1) importance degrees are previously determined 
among all advertisements and the display priorities con- 
form to the importance degrees, (2) when the vector 
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matching of the user's individual information and the ad- 
vertisement is taken, the matching degree of both is cal- 
culated and the display priorities are determined in order 
of the matching degree, and (3) the methods (1 ) and (2) 
are combined so that the importance degrees are 
weighted in accordance with the matching degrees and 
the weighted importance degrees are used as the dis- 
play priorities. In the embodiment, the method (1 ) is sup- 
posed. Definitely, as shown in Fig. 24, an item 487 of 
the importance degrees is provided in the advertisement 
list and the display priorities are determined in order of 
advertisements having higher importance degrees. For 
example, in Figs. 23 and 24, the display priorities are in 
order of advertisements 470, 471, 472, 473, 474, 475, 
476 and 477. Further, when the importance degrees are 
the same, the display priorities are determined on the 
basis of other factors, while in the embodiment, the dis- 
play priorities are determined simply in order of descrip- 
tion in the advertisement list. 

The importance degree corresponds to the adver- 
tisement charge. Accordingly, an advertisement of 
goods to which increased advertisement charge is paid 
is displayed preferentially. 

In next step 438, the advertisements are embedded 
in the home page in order of the priority. A conceptual 
structure of the home page in which advertisements are 
embedded is shown in Fig. 25. The home page 490 in- 
cludes a content portion 491 and advertisements ar- 
ranged in order of the display priority. The reason why 
the structure of the home page is described to be con- 
ceptual is that a home page generally has only informa- 
tion displayed simultaneously and when display of a part 
of the home page such as, for example, the advertise- 
ment area is desired to be changed, an inquiry is made 
to the server any time. However, improvement of the 
current WWW can cope with the structure shown in Fig. 
25 and accordingly in the embodiment description is 
made as it is. 

Finally, in step 439, the home page 490 is transmit- 
ted to the user terminal. 

Next, a flow (Fig. 22) of processing in the user ter- 
minal is described. 

First, in step 450, the WWW browser transmits a 
request command of the home page to the WWW serv- 
er. At the same time, the WWW browser sends the in- 
dividual information to the WWW server in cooperation 
with the individual information management program. 
The contents server receives the information and per- 
forms the processing shown in Fig. 21 . Accordingly, the 
WWW browser waits until the home page is completely 
received (step 451 ). 

In next step 452, the home page is displayed. At this . 
time, the advertisement is also displayed in accordance 
with the display priority besides the contents. The dis- 
play picture in this case is as shown in Fig. 20, for ex- 
ample. " 

The processing in steps 453 and 454 are repeated 
even/predeterminedtime. In step 453, whether another 
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advertisement is displayed or not is examined. Since the 
duration of display of each advertisement is previously 
prescribed, the timer management program examines 
the duration. When it is time to change the advertise- 
ment, operation is returned to step 452 in which only the 
advertisement is changed to an advertisement having a 
next display priority. Otherwise, operation is moved to 
step 454. 

When it is assumed that the duration also corre- 
sponds to the advertisement charge, the advertisement 
of goods to which the increased advertisement charge 
is paid can be displayed for a long time. 

In next step 454, whether a request for changing 
contents is issued from the user or not is examined. 
When there is the request for changing contents, oper- 
ation is returned to step 450 and the operation starting 
trom the transmission of the content request command 
is repeated. Otherwise, operation is returned to step 
453. 

The method of providing the information of goods 
in the WWW on the Internet has been described, while 
the present invention is not limited to the Internet and 
the WWW and can be used in other communication me- 
dia such as, for example, the communication using the 
personal computer. 

According to the above embodiments, the exist- 
ence of data classified or individualized for each user in 
relation to the displayed image scene can be notified to 
the user by the picture or character string indicative of 
contents of the relative data. At the same time, the rel- 
ative data can be displayed in synchronism with the im- 
age in response to a request of the user In this manner, 
the possibility of accessing from the image to the relative 
data simply brings the information provider a merit that 
the viewers are drawn in by a broadcasting image to 
move to the bidirectional media such as the Internet im- 
mediately. At the same time, the viewer can know de- 
tailed information relative to the image more simply. Fur- 
ther, since information suitable for the individual user 
can be sent directly, information can be provided and 
referred effectively. 

Further, according to the embodiments, since the 
display condition and the display order of the picture and 
the character string indicative of the existence of the rel- 
ative data are determined on the basis of the utilization 
environment condition, the existence of the relative data 
can notified to the user clearly and understandably. 

Furthermore, in the embodiments, since there is 
provided the mode for displaying the image and the data 
simultaneously, access can be made to the image and 
the data freely to refer to information. 

In addition, when the user inputs a command for dis- 
playing a list, a list of pictures and character strings dis- 
played until now is displayed. When the user selects any 
one in the list, the relative data is displayed. Accordingly, 
the user can first view only the images continuously and 
refer to the relative data slowly later. 

Moreover, in the embodiments, the number of rela- 



tive data to be displayed as a list is limited and when the 
number of data exceeds the limitation, the data is se- 
lected rn accordance with the priority. Accordingly, the 
memory area can be maintained to be small and conse- 

s quently the cost of the terminal can be reduced. 

Further, in the embodiments, since there is provided 
the mode for forcedly displaying the picture and the 
character string indicative of contents of the relative da- 
ta, the intention of the information provider can be trans- 

10 mitted to the user of the target exactly. 



Claims 

is 1 . An image and data display method of displaying an 
image and at least one data relative to a specific 
scene of said image, comprising the steps of: 

(a) determining a display condition (18) includ- 
ing display and non-display of relative data on 
basis of a previously determined user identifier 
and utilization condition (17); 

(b) displaying a picture and a character string 
indicative of contents of at least one relative da- 
ta determined to be able to be displayed in said 
step (a) on basis of said display condition to- 
gether with the image during display of said 
specific scene; 

(c) displaying, when at least one of said pic- 
tures and character strings is selected, relative 
data corresponding to said selected picture and 
character string on basis of said display condi- 
tion; and 

(d) displaying, when a command for displaying 
a list is inputted, a list of picture and character 
strings displayed until now and, when at least 
one in said list is selected, displaying relative 
data corresponding to said selected picture and 
character string on basis of said display condi- 
tion. 



25 



30 



35 



40 

2. An image and data display method of displaying an 
image and at least one data relative to a specific 
scene of said image, comprising the steps of: 

45 

(a) determining a display condition (18) of rel- 
ative data on basis of a previously determined 
user identifier and utilization condition (17); 

(b) displaying a picture and character string in- 
dicative of contents of at least one relative data 
on basis of said display condition together, with 
the image during display of said specific scene; 

(c) displaying, when at least one of said pic- 
tures and character strings is selected, relative 
data corresponding to said selected picture and 
character string on basis of said display condi- 
tion; and 

(d) displaying, when a command for displaying 
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a list is inputted, a list of pictures and character 
strings displayed until now and, when at least 
one in said list is selected, displaying relative 
data corresponding to said selected picture and 
character string on basis of said display condi- s 
tion. 

An image and data display method of displaying an 
image and at least one data relative to said image, 
comprising the steps of: to 

(a) selecting, when one of a plurality of images 
is selected, at least one data relative to said im- 
age simultaneously; 

(b) determining a display condition (18) includ- *5 
ing display and non-display of said relative data 

on basis of a previously determined user iden- 
tifier and utilization condition (17); and 

(c) displaying at least one relative data deter- 
mined to be able to be displayed in said step 20 
(b) in accordance with said display condition in 
synchronism with said image. 

An image and data display method of displaying an 
image and at least one data relative to a specific 2s 
scene of said image, comprising the steps of: 

(a) selecting, when one of a plurality of images 
is selected, at least one data relative to said im- 
age simultaneously; 30 

(b) determining a display condition (18) includ- 
ing display and non-display of said relative data 
on basis of a previously determined user iden- 
tifier and utilization condition (17); 

(c) displaying a picture and character string in- 35 
dicative of contents of at ieast one relative data 
determined to be able to be displayed in said 
step (b) on basis of said display condition to- 
gether with the image during display of said 
specific scene; -to 

(d) displaying, when at least one of said pic- 
tures and character strings is selected, relative 
data corresponding to said selected picture and 
character string on basis of said display condi- 
tion; and 45 

(e) displaying, when a command for displaying 
a list is inputted, a list of pictures and character 
strings displayed until now and, when at least 
one in said list is selected, displaying relative 
data corresponding to said selected picture and so 
character string on basis of said display condi- 
tion. 

An image and data display method of displaying an 
image and data relative to a specific scene of said 55 
image, comprising the steps of: 

(a) selecting, when one of a plurality of images 



is selected, at least one data relative to said im- 
age simultaneously; 

(b) determining a display condition (18) inciua- 
ing display and non-display of said relative data 
on basis of a previously determined user iden- 
tifier and utilization condition (17); 

(c) determining, when a plurality of relative data 
are determined to be able to be displayed in 
said step (b), display priorities among said rel- 
ative data; 

(d) displaying a picture and character string in- 
dicative of contents of each relative data on ba- 
sis of said display priorities determined in said 
step (c) together with the image during display 
of said specific scene; 

(e) displaying, when a picture and character 
string is selected, relative data corresponding 
to said selected picture and character string on 
basis of said display condition; and 

(f) displaying, when a command for displaying 
a list is inputted, a list of pictures and character 
strings displayed until now and, when at least 
one in said list is selected, displaying relative 
data corresponding to said selected picture and 
character string on basis of said display condi- 
tion. 

6. An image and data display method according" to 
Claim 5, wherein in said step (e) a display screen 
is divided into at least one image display area and 
at least one data display area in display of relative 
data and a mode for simultaneously displaying an 
image and data relative to said image and a mode 
for displaying at least one relative data on a whole 
display screen are provided so that a display mode 
is changed among an image display mode, said im- 
age and data simultaneous display mode and said 
data display mode when a command for change- 
over of display is inputted. 

7. An image and data display method according to 
Claim 6, wherein said step (e) comprises, in said 
image and data simultaneous display mode, the 
steps of: 

(el ) displaying a picture and character string 
indicative of contents of each relative data in 
said image display area on basis of said display 
priority determined in said processing and said 
display condition together with the image dur- 
ing display of said specific scene; 
(e2) displaying, when said picture and charac- 
ter string is selected, relative data correspond- 
ing to said selected picture and character string 
in said data display area on basis of said display 
condition; and 

(e3) displaying, when a command for display- 
ing a list is inputted, a list of pictures and char- 
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acter strings displayed until now in said data 
display area and, when at least one in said list 
is selected, displaying relative data corre- 
sponding to said selected picture and character 
string in said data display area on basis of said s 
display condition. 

8. An image and data display method according to 
Claim 7, wherein in said step (d) when there are a 
picture and character string of relative data which 10 
cannot be displayed since said display priority 
thereof is low, a list including pictures and character 
strings displayed until now and the picture and char- 
acter string which cannot be displayed is displayed 
when the command for displaying the list is inputted is 
in said step (f) and when at least one in said list is 
selected, relative data corresponding to said select- 
ed picture and character string is displayed on basis 

of said display condition. 

20 

9. An image and data display method according to 
Claim 8, wherein in said step (f) number of relative 
data displayed in said list is limited and when said 
limitation is exceeded, relative data is selected in 
accordance with said display priority. 25 

10. An image and data display method according to 
Claim 9, wherein there is provided a mode in which 
a picture and character string indicative of contents 

of relative data can be set by a user not to be dis- 30 
played, and when the command for displaying the 
list is inputted even in said non-display mode, the 
list of pictures and character strings displayed until 
now is displayed. 

35 

11. An image and data display method according to 
Claim 8, wherein there is provided a mode in which 
an information provider can forcedly display a pic- 
ture and character string indicative of contents of 
relative data, and even when a user sets the picture 40 
and character string of relative data not to be dis- 
played, the picture and character string in the rela- 
tive data in said forced display mode are necessar- 
ily displayed. 

45 

12. An image and data display apparatus including 
means (14,3) for inputting and displaying an image, 
means (15, 3) for inputting and displaying data rel- 
ative to an image, means for selecting an image in 
response to a request from a user, and means (29) so 
for changing to display an image or data in response 

to a request from the user, comprising: 

means ( 1 6) for selecting data on basis of a pre- 
viously registered user identifier and utilization ss 
condition; 

means (21) for controlling synchronization of 
display of an image and data on basis of syn- 



chronizing information; 

means for solving competition in displaying a 
plurality of data simultaneously; and 
means (22) for combining data with an image 
or the image with the data on basis of said syn- 
chronizing information. 

13. An image and data display apparatus including dis- 
play means (3) for displaying an image or data, con- 
trol means (2) for receiving an image including rel- 
ative data externally and supplying said image to 
said display means, and input means (4) for indicat- 
ing display to said control means, said control 
means (2) comprising: 

receiving means (14) for selecting an external 
image on basis of an indication of said input 
means; 

data selection means ( 1 6) for selecting data rel- 
ative to said selected image on basis of an in- 
dication of said input means; 
synchronization control means (21) for syn- 
chronizing display of said selected data with 
said selected image; and 

combination display means (22) for combining 
said selected image with data synchronized 
with said image on basis of an indication of said 
input means to supply it to said display means. 
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